The Landmark Selection Method for Multiple Output Prediction

Conditional modeling x 7→ y is a central problem in machine learning. A substantial research effort is devoted to such modeling when x is high dimensional. We consider, instead, the case of a high dimensional y, where x is either low dimensional or high dimensional. Our approach is based on selecting a small subset yL of the dimensions of y, and proceed by modeling (i) x 7→ yL and (ii) yL 7→ y. Composing these two models, we obtain a conditional model x 7→ y that possesses convenient statistical properties. Multi-label classification and multivariate regression experiments on several datasets show that this method outperforms the one vs. all approach as well as several sophisticated multiple output prediction methods.